Papers with multi-modal alignment
CONE: An Efficient COarse-to-fiNE Alignment Framework for Long Video Temporal Grounding (2023.acl-long)
Copied to clipboard
Zhijian Hou, Wanjun Zhong, Lei Ji, Difei Gao, Kun Yan, W.k. Chan, Chong-Wah Ngo, Mike Zheng Shou, Nan Duan
| Challenge: | Existing work on video temporal grounding for long videos is limited by existing datasets. |
| Approach: | They propose a query-guided window selection strategy and a coarse-to-fine mechanism to speed up inference for long videos. |
| Outcome: | The proposed framework accelerates inference time by 2x on Ego4D-NLQ and 15x on MAD while keeping SOTA results. |
Rethinking Text-based Protein Understanding: Retrieval or LLM? (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies have focused on integrating protein-related knowledge into large language models through continued pretraining and multi-modal alignment. |
| Approach: | They propose a retrieval-enhanced method which significantly outperforms fine-tuned LLMs for protein-to-text generation and shows accuracy and efficiency in training-free scenarios. |
| Outcome: | The proposed method significantly outperforms fine-tuned LLMs for protein-to-text generation and shows accuracy and efficiency in training-free scenarios. |
Breaking the Noise Barrier: LLM-Guided Semantic Filtering and Enhancement for Multi-Modal Entity Alignment (2025.emnlp-main)
Copied to clipboard
| Challenge: | Multi-modal entity alignment (MMEA) aims to identify equivalent entities between two multimodal knowledge graphs. |
| Approach: | They propose a novel LLMguided MMEA framework that prioritizes noise reduction before fusion. |
| Outcome: | The proposed framework prioritizes noise reduction before fusion and improves semantics on the noisy FB YG dataset. |